1. Field of the Invention
The present invention relates to imaging technology, color imaging technology, digital color imaging technology, correction of color imbalances in digital color images, and blended manual/software systems for correcting such color imbalances.
2. Background of the Art
The proliferation of digital imaging means such as photography, scanning, copying, printing and digital cameras has resulted in a large volume of color imagery being generated. As no device produces consistently perfect color, especially in the hands of unskilled amateurs, there is a need to correct the color of the images ultimately provided. In particular, it is often required that a specific color or range of colors is rendered correctly. This is especially true of so called “memory” colors, such as skin tones and the colors of the sky, vegetation and certain foodstuffs (e.g. bananas, lemons, oranges, strawberries, red apples, etc.). Humans have a very strong sense of when such colors are correct, and an image with otherwise good color balance can be rejected as unsatisfactory if these particular tones are not judged to be perfect. Other situations where the correctness of specific colors is critical includes colors associated with product brands, where logos and packaging must maintain a consistent look across many images even at the expense of some slight shifts in other colors. It is also acknowledged that different cultures have shown preferences for different color balances or saturation levels, based on exposure to particular color schemes during culturalization.
It is well know to correct the color balance of an entire image and a number of methods are used, most of which provide some level of automation. One method corrects by recovering the color of the illuminant, as exemplified by U.S. Pat. No. 4,685,071, U.S. Pat. No. 5,495,428, U.S. Pat. No. 5,825,916 or U.S. Pat. No. 6,104,830. The illuminant can also be recovered from the gamut of colors in the image as disclosed in European Pat. No. 862,336. Another approach is based the assumption that the colors of an image average to gray. The original patent in this area is U.S. Pat. No. 2,571,697 and improvements are provided in U.S. Pat. No. 5,233,413; U.S. Pat. No. 5,357,352 and U.S. Pat. No. 5,420,704. It is also known in the art to use statistical analyses of the properties of commonly encountered images to determine optimal whole image color correction. This is taught in include U.S. Pat. No. 4,339,517 and U.S. Pat. No. 6,097,836 and, in a related way, in U.S. Pat. No. 5,694,484 and WO 97/01151. There are additional correction procedures of a more or less empirical nature such as those of U.S. Pat. No. 4,729,016, U.S. Pat. No. 4,984,071, U.S. Pat. No. 5,117,293, U.S. Pat. No. 5,323,241, U.S. Pat. No. 5,371,615 and U.S. Pat. No. 6,151,410. Related correction procedures are provided in software by the “Auto Levels” feature of Photoshop 5.5 (Adobe Systems Incorporated, 345 Park Avenue, San Jose, Calif. 95110-2704) or the “Auto Tonal Adjustment” feature of PhotoStyler 2.0 (Aldus Corporation, 411 First Avenue South, Seattle, Wash. 98104).
It is also well know to correct the color balance of an image manually. For example, an operator may specify a black or white point in the image or a color that should be considered as neutral gray. Such a capability is available as software in the “Curves” feature of Photoshop 5.5 (Adobe Systems Incorporated, 345 Park Avenue, San Jose, Calif. 95110-2704), the “Balance to sample” feature of PhotoStyler 2.0 (Aldus Corporation, 411 First Avenue South, Seattle, Wash. 98104), and in the “Automatic” mode of the “Tint” feature in PhotoDraw 2000 (Microsoft Corporation, One Microsoft Way, Redmond, Wash. 98052-6399). Correction using manually specified highlight and shadow regions is disclosed in U.S. Pat. No. 5,062,058 and the falls within the claims of U.S. Pat. No. 5,487,020. However, neither these methods nor the aforementioned automated methods provide a means of correcting a specific color in an image.
Local color modification in an image can be accomplished by restricting the region of the image in which changes take place. U.S. Pat. No. 5,943,059 teaches a method of modifying the colors of elements of a business graphic, which are specified by means of an index of elements within the image. However, this method is not applicable to general imagery and, even in the restricted area of business graphics, is of only a slight benefit if the image does not contain a pre-existing index of elements. U.S. Pat. No. 5,742,520 discloses color correction by means of detection of approximately 10 to 12 color categories within an image. Such a method is insufficiently exact for images containing several million colors or more and cannot be used to denote a specific critical color for correction, being restricted to pre-determined color categories. U.S. Pat. No. 6,016,168 discloses a method of strengthening or weakening a specific color in an image based on its location in a region of RGB chromaticity space and a pre-established set of correction factors for these regions. This procedure does not permit the operator to choose the color to be corrected nor to select a replacement color for the color being modified and is, moreover, incapable of differentiating colors by their brightness.
A number of other methods for local color modification are known in the image editing art. One prior art procedure is to select the region to be modified using commonly available freehand selection tools such as a “shaped selection tool” (e.g. a rectangle or circle), the “lasso tool” or “point to point selection tool” as found in image editing software such as Paint Shop Pro 7 (Jasc Software Inc., 7905 Fuller Road, Eden Prairie, Minn., 55344). Another prior art method is to select an image region for modification according to color range. This can be accomplished by the method of “magic wand” selection in Paint Shop Pro 7 and other image editors or by the related procedure taught in U.S. Pat. No. 5,506,946. In addition to being laborious, these selection methods result in a discrete edge to region being corrected. After the correction the abrupt change in color at this edge can result in an unnatural look to the image. While it is know to anti-alias, feather or otherwise blur such edges, the optimal amount of such blurring is image dependent and requires trial and error to determine. U.S. Pat. No. 5,506,946 teaches a fixed amount of smoothing that will not be appropriate for arbitrary images in a range of sizes. U.S. Pat. No. 6,128,001 discloses a related procedure wherein the region of the image to be modified is defined by an alpha matte. A third prior art method of modifying color is by means of a brush that is passed over a region of the image to be altered. The majority of such brushes have the disadvantage that lightness and saturation variation within the region being modified are replaced by a single uniform and invariant color. However, it is also possible to retain such texture detail in the modified area of the image by means of a special brush. An example is the “retouch tool” of Paint Shop Pro 7 in the “hue to target” or “color to target” mode. The brush is controlled by means of a pointing device such as a mouse, trackball or pen and tablet, and can be integrated with a touch-screen. Considerable dexterity and skill is required to successfully modify an image in this way. Furthermore, an operator must have the necessary experience to properly set brush characteristics so that the region of modified color blends undetectably with the rest of the unmodified image.
ColorPilot® 4.10 (Invention office RG, kv.7, d.8, ul. Shkolnaia, Kokoshkino, Moskovskaia obl. 143390, Russian Federation) is a software program offering an alternative approach to color correction. The operator first roughly marks a region of an image as containing a source color to be modified and then selects a target color that will replace the source color. In consequence, the average source color is, in principle, converted to the average target color while other colors in the image undergo an intermediate and appropriate change. However, in practice, though the procedure works well with colors of medium brightness, the use of this type of method with colors of low brightness or high brightness gives very poor results. The latter (high brightness) is particularly important because skin tones are often of high brightness and their proper correction is critically important to the viewer of the image. The former (low brightness) can lead to unpleasant and muddy blacks. Color is conveniently characterized by hue, brightness and saturation (or equivalents thereof) of which hue is most significant, determining, for instance, whether a color is considered green or blue or whether one color is more red than another. ColorPilot® permits transfer of all three of these characteristics from target to source, or of brightness only, or of a combination of hue and saturation. However, there is no method for independently transferring from target to source only hue, the chief attribute of color, or combinations of only hue and brightness. This severely limits the ability to correct the color and as a result can yield unnatural color shifts in the modified image. Additionally, even when only the hue and saturation of the target color are applied to the image, the brightness is in fact observed to change, undesirably disrupting the contrast characteristics of the image.
U.S. Pat. No. 6,058,208 claims a color correction device for correcting colors in a color image, comprising: standard point registration means for registering coordinates in a color space of at least one standard point, the color space having axes for specifying colors and a standard point representing a color to be used as a standard for color correction; object point registration means for registering coordinates in the color space of an object point for each standard point, each object point representing a desired color to be achieved by color correction of a standard point; and correction standard indication means for receiving an indication of at least two pairs of a standard point and an object point that are arbitrary points in the color space, and for registering standard points and object points of at least two pairs in the standard point registration means and the object registration means respectively; and coordinate conversion means for determining a single mapping function which simultaneously maps all of the standard points registered in the standard point registration means to respective corresponding object points registered in the object point registration means and which can be used for mapping any arbitrary point in the color space, and for generating a corrected color image from an inputted pre-correction image using the mapping function. Thus, this invention, like ColorPilot®, uses source and target colors but requires not one but two pairs of such colors. To a person seeking to correct one specific color, the need to select a second color is redundant and experience shows that simultaneous correction of two colors is more difficult for an unskilled operator than if the operator can focus only on the single color to be corrected. Moreover, the method disclosed requires a complex correction to be computed iteratively using weighting of colors by their distance from the source colors and, as a result, is undesirably slow. U.S. Pat. No. 5,487,020 claims a method for color correcting a predetermined portion of a color image, said color image comprising a plurality of pixels and a plurality of color information channels for each of said plurality of pixels, said method comprising the steps of: selecting in said predetermined portion a first input reference color of a first pixel and a corresponding first desired output color of the first pixel; selecting in said predetermined portion a second input reference color of a second pixel and a corresponding second desired output color of the second pixel; defining a linear color transformation for each color information channel based on the input reference colors and the desired output colors; applying said linear color transformation to all pixels in said predetermined portion; and displaying the color image. This procedure, again requiring two pairs of source and target colors, is not suited to the correction of a single color and, by virtue of linear correction, does not permit independent choice of hue, brightness and saturation.